Multi-stage Suture Detection for Robot Assisted Anastomosis based on Deep Learning

In robotic surgery, task automation and learning from demonstration combined with human supervision is an emerging trend for many new surgical robot platforms. One such task is automated anastomosis, which requires bimanual needle handling and suture detection. Due to the complexity of the surgical environment and varying patient anatomies, reliable suture detection is difficult, which is further complicated by occlusion and thread topologies. In this paper, we propose a multi-stage framework for suture thread detection based on deep learning. Fully convolutional neural networks are used to obtain the initial detection and the overlapping status of suture thread, which are later fused with the original image to learn a gradient road map of the thread. Based on the gradient road map, multiple segments of the thread are extracted and linked to form the whole thread using a curvilinear structure detector. Experiments on two different types of sutures demonstrate the accuracy of the proposed framework.Comment: Submitted to ICRA 201

Feature extraction from ear-worn sensor data for gait analysis

Gait analysis has a significant role in assessing human's walking pattern. It is generally used in sports science for understanding body mechanics, and it is also used to monitor patients' neuro-disorder related gait abnormalities. Traditional marker-based systems are well known for tracking gait parameters for gait analysis, however, it requires long set up time therefore very difficult to be applied in everyday realtime monitoring. Nowadays, there is ever growing of interest in developing portable devices and their supporting software with novel algorithms for gait pattern analysis. The aim of this research is to investigate the possibilities of novel gait pattern detection algorithms for accelerometer-based sensors. In particular, we have used e-AR sensor, an ear-worn sensor which registers body motion via its embedded 3-D accelerom-eter. Gait data was given semantic annotation using pressure mat as well as real-time video recording. Important time stamps within a gait cycle, which are essential for extracting meaningful gait parameters, were identified. Furthermore, advanced signal processing algorithm was applied to perform automatic feature extraction by signal decomposition and reconstruction. Analysis on real-word data has demonstrated the potential for an accelerometer-based sensor system and its ability to extract of meaningful gait parameters

Image Perception

n/

A Multi-Robot Cooperation Framework for Sewing Personalized Stent Grafts

This paper presents a multi-robot system for manufacturing personalized medical stent grafts. The proposed system adopts a modular design, which includes: a (personalized) mandrel module, a bimanual sewing module, and a vision module. The mandrel module incorporates the personalized geometry of patients, while the bimanual sewing module adopts a learning-by-demonstration approach to transfer human hand-sewing skills to the robots. The human demonstrations were firstly observed by the vision module and then encoded using a statistical model to generate the reference motion trajectories. During autonomous robot sewing, the vision module plays the role of coordinating multi-robot collaboration. Experiment results show that the robots can adapt to generalized stent designs. The proposed system can also be used for other manipulation tasks, especially for flexible production of customized products and where bimanual or multi-robot cooperation is required.Comment: 10 pages, 12 figures, accepted by IEEE Transactions on Industrial Informatics, Key words: modularity, medical device customization, multi-robot system, robot learning, visual servoing, robot sewin